RNN model
tf.keras.Sequential
임베딩층-RNN층-완전연결층
RNN층 종류
SimpleRNN: 완전 연결 순환 층인 기본 RNN
LSTM: 긴 의존성을 감지할 수 있는 LSTM RNN
GRU: LSTM의 대앙인 GRU 유닛을 사용한 순환 층
numpy 버전 오류로 인해서 SimpleRNN Sequential에 추가 불가능
GIOVANNIN로 진행
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Embedding
from tensorflow.keras.layers import SimpleRNN
from tensorflow.keras.layers import Dense
model=Sequential()
model.add(Embedding(input_dim=1000, output_dim=32))
model.add(SimpleRNN(32, return_sequences=True))
model.add(SimpleRNN(32))
model.add(Dense(1))
model.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
embedding (Embedding) (None, None, 32) 32000
_________________________________________________________________
simple_rnn (SimpleRNN) (None, None, 32) 2080
_________________________________________________________________
simple_rnn_1 (SimpleRNN) (None, 32) 2080
_________________________________________________________________
dense (Dense) (None, 1) 33
=================================================================
Total params: 36,193
Trainable params: 36,193
Non-trainable params: 0
_________________________________________________________________
Build Model & compile & fit
embedding_dim=20
vocab_size=len(token_counts)+2
tf.random.set_seed(1)
bi_lstm_model=tf.keras.Sequential([
tf.keras.layers.Embedding(input_dim=vocab_size, output_dim=embedding_dim, name='embed-layer'),
tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(64, name='lstm-layer'), name='bidir-lstm'),
tf.keras.layers.Dense(64, activation='relu'),
tf.keras.layers.Dense(1, activation='sigmoid')])
bi_lstm_model.summary()
Model: "sequential_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
embed-layer (Embedding) (None, None, 20) 1740180
_________________________________________________________________
bidir-lstm (Bidirectional) (None, 128) 43520
_________________________________________________________________
dense_1 (Dense) (None, 64) 8256
_________________________________________________________________
dense_2 (Dense) (None, 1) 65
=================================================================
Total params: 1,792,021
Trainable params: 1,792,021
Non-trainable params: 0
_________________________________________________________________
Bidirectional
Bidirectional 래퍼 클래스는 입력 시퀀스를 정방향과 역방향으로 두번 처리한다.(역전파의 정방향 계산이 아님)
기본적으로 정방향과 역방향으로 처리한 결과는 연결된다.
merge_mode를
sum, mul, ave, concat(기본값), None으로 지정하여 두 텐서를 리스트로 반환할 수 있다.
bi_lstm_model.compile(optimizer=tf.keras.optimizers.Adam(1e-3), loss=tf.keras.losses.BinaryCrossentropy(from_logits=False), metrics=['accuracy'])
history=bi_lstm_model.fit(train_data, validation_data=valid_data, epochs=10)
test_results=bi_lstm_model.evaluate(test_data)
print('테스트 정확도: {:.2f}%'.format(test_results[1]*100))